1. Field of the Invention
The present disclosure generally relates to the field of advertisement placement engines.
2. Related Art
In e-commerce, an advertisement placement engine delivers a list of deals or advertisements to a user. The deals may be selected from an inventory of hundreds of millions of deals from merchants. The selection criteria are based on the keywords, categories or a combination of keywords and categories entered by the user. When the user clicks or buys one of the deals, the engine is credited with a commission (or bid) from the merchant, which is known as a yield. A key performance objective is to optimize the yield, which can be achieved by optimizing two factors, namely, relevance and cost per click (CPC). Relevance represents the degree of how closely the search for deals matches the user's intent, and CPC represents how much bid from merchant can be obtained for the user selecting a particular deal. The higher the relevance, the higher the click through rate (CTR), and the higher the bid, the higher the CPC.
An advertisement placement engine has to balance the relevance with CPC, and in this respect, is different from a general purpose search engine that focuses on relevance only. The selection of deals by the engine starts with a relevance search, which retrieves a list of relevant deals based on keywords, categories, user profile, and any other information. In order to enhance the CPC, the bid in each deal is typically used to influence the final ordering of the deal list, using a re-ranking yield sort algorithm. For example, if two deals have similar relevance score, the engine may choose to place the deal with a higher bid value ahead. If the yield sort algorithm leans too much on the bid value, the relevance may suffer, which in turn would affect the CTR of the deals. To further complicate matters, the engine may further search deals across more merchants to give the customer better merchant selections. In this case, a merchant diversity algorithm is also used to demote finding deals from the same merchant.
The advertisement placement engine typically processes a user query in the following order: conduct a lexical match of keywords with deals in the corpus (a large collection of 100 million plus deals); pick the top 40,000 deals with the best lexical and static score; filter deals by using categories and user profile influence; re-rank deals based on a yield sort algorithm; and re-rank deals again based on a merchant diversity algorithm.
The typical methods used in the art to tune and optimize the engine are human judgment and the AB test, both of which have limitations. The method of human judgment uses a human judging process to deter mine the Normalized Discounted Cumulative Gain (NDCG) of a result set from a sample set of keywords sent to the target engine. The higher the NDCG, the higher the expected relevance. The human judgment process, however, is slow with the turnaround time for several thousands of queries being many hours or sometimes days, and expensive. With the typical cost to judge a query being about 0.4 dollars, the total cost could be very high when the testing requires judging thousands of queries. Further, in this method, the process cannot be automated, and greatly incurs development time.
On the other hand, the method of AB test channels a small percentage of the production traffic to the target engine, and measures the CTR and CPC. The higher the CTR and CPC produced by the target engine, the higher the expected CPC. The AB test, too, is slow, usually requiring days of testing to collect enough data to judge the CTR and CPC of the test, and expensive because the AB testneeds to expose part of the production traffic to an untested engine, which may directly impact the business negatively. Further, if the test result is not desirable, the development team may be left with little time to trouble shoot the problem and forced to cancel or delay the beta release. In that case, an additional step prior to the AB test is needed to predict the quality. Thus, each AB test takes a long time and incurs huge expenses and opportunity cost.
Therefore, it is desirable to provide a system and method to tune and optimize a target advertisement placement engine that is fast as well as inexpensive as compared to the conventional tuning tools.